Capturing a misalignment

ABSTRACT

Systems and method for providing misaligned image features. A method includes receiving a first and a second image data set, wherein the first and the second image data sets map at least partially a shared examination region of an examination object, registering the first image data set with the second image data set, determining a distance data set based on the registered first image data set and the second image data set, identifying the misaligned image features in the distance data set that are caused by a misalignment between the registered first and the second image data sets, and providing the identified misaligned image features.

CROSS REFERENCE TO RELATED APPLICATIONS

This patent document also claims the benefit of DE 102019217576.7 filedon Nov. 14, 2019 which is hereby incorporated in its entirety byreference.

FIELD

Embodiments relate to a computer-implemented method for capturingmisaligned image features.

BACKGROUND

X-ray based imaging methods are often used for capturing changes overtime at an examination region of an examination object, for example ahuman and/or an animal patient. In this connection the change over timeat the examination region of the examination object may include, forexample, a spreading movement and/or flowing movement of a contrastmedium in a vessel system and/or a movement of a medical object, forexample a medical instrument, for example a catheter and/or guide wire,and/or a diagnostic instrument, for example an endoscope. The X-raybased imaging methods frequently include digital subtraction angiography(DSA) where at least two chronologically recorded X-ray images, that mapat least partially the shared examination region, are subtracted fromeach other. A differential image is frequently provided as a result ofthe DSA. The components in the differential image that are irrelevantand/or disruptive to a treatment and/or diagnosis, and for example donot change over time, may frequently be reduced and/or removed hereby.

From the prior art it is also known that before the subtraction of theat least two X-ray images, registering between the at least two X-rayimages relative to each other occurs. While movement artifacts and/oraltered recording parameters, for example an altered recording geometry,may be at least partially compensated hereby, it is disadvantageous,however, that in the case of inadequate registering between the at leasttwo X-ray images, image artifacts may frequently occur in thedifferential image, that remain unknown and/or may be misinterpreted fora diagnosis and/or treatment.

BRIEF SUMMARY AND DESCRIPTION

The scope of the present invention is defined solely by the appendedclaims and is not affected to any degree by the statements within thissummary. The present embodiments may obviate one or more of thedrawbacks or limitations in the related art.

Embodiments analyze the reliability of registering of medical images insuch a way that subsequent medical image interpretation is supported.

Embodiments are described below in relation to both methods andapparatuses for providing misaligned image features and in relation tomethods and apparatuses for providing trained functions. Features,advantages, and alternative embodiments of data structures and/orfunctions in the case of methods and apparatuses for providingmisaligned image features may be transferred here to analogous datastructures and/or functions in methods and apparatuses for providingtrained functions. Analogous data structures may be identified by theuse of the prefix “training”. The trained functions used in methods andapparatuses for providing misaligned image features may include beenadjusted and/or provided for example by methods and apparatuses forproviding trained functions.

Embodiments relate to a computer-implemented method for providingmisaligned image features. A first and a second image data set arereceived. The first and second image data sets map at least partially ashared examination region of an examination object. The first image dataset is registered with the second image data set. A distance data set isdetermined on the basis of the registered first image data set and thesecond image data set. The misaligned image features in the distancedata set are identified, that are caused by a misalignment between theregistered first and the second image data set. The identifiedmisaligned image features are provided.

Receiving of the first and/or the second image data set may include, forexample, acquisition and/or reading out from a computer-readable datamemory and/or receiving from a data memory unit, for example a database.The first and/or the second image data set may be provided by aproviding unit of a medical imaging device.

The first and/or second image data set may include, for example,two-dimensional and/or three-dimensional image data, including aplurality of image points, for example pixels and/or voxels. The firstand second image data sets map at least partially a shared examinationregion of the examination object. In addition, the first and/or secondimage data set may map a course over time, for example a phase of thecourse over time, of a change at the examination region of theexamination object. The phase of the course over time of the change atthe examination region of the examination object may be based, forexample, on a physiological parameter of the examination object, forexample a respiratory state and/or a cardiac state, and/or a phase of aspreading movement and/or flowing movement of a contrast medium in avessel system of the examination object and/or a phase of a movement ofa medical object arranged in the examination region.

With digital subtraction angiography (DSA), for example the first imagedata set may map a mask phase, while the second image data set may map afull phase. The first image data set and/or the second image data setmay include, for example, in each case at least one projection X-rayimage of the examination region of the examination object. For example,the first and/or second image data set may include a two-dimensionaland/or three-dimensional scene.

The first and second image data sets may be recorded by a medicalimaging device. The medical imaging device may be configured as an X-raydevice and/or a C-arm X-ray device and/or magnetic resonance system(MRT) and/or computed tomography system (CT) and/or sonography systemand/or positron emission tomography system (PET).

The first and/or second image data set may include metadata. Themetadata may include information on recording parameters and/oroperating parameters of the medical imaging device for recording of thefirst and/or of the second image data set.

When registering the first image data set with the second image dataset, the first image data set may be rigidly and/or non-rigidlytransformed according to a transformation rule. The first image data setmay be registered in relation to the second image data set according tothe transformation rule. The second image data set may be registered inrelation to the first image data set according to the transformationrule. Registering may take place in such a way that the section, mappedin the first and in the second image data sets, of the at leastpartially shared examination region is optimally correlated. Registeringof the first image data set with the second image data set may also takeplace on the basis of the metadata of the first and/or of the secondimage data set. For example, registering may include providing aregistered first image data set.

The distance data set may be determined on the basis of the registeredfirst image data set and the second image data set provided. Thedistance data set may include information relating to a differenceand/or a distance between at least one first region-of-interest (ROI),including a plurality of image points of the registered first image dataset, and a second region-of-interest corresponding therewith, includinga plurality of image points of the second image data set. The distancedata set may also include information, for example two-dimensionaland/or three-dimensional information, relating to a registering errorbetween the at least one first region-of-interest and the secondregion-of-interest corresponding therewith. Determination of thedistance data set may include a difference and/or a scalar productand/or a comparison between the registered first image data set and thesecond image data set. The distance data set may be determined in astandardized manner and/or be standardized.

The distance data set may exhibit all differences between the registeredfirst image data set and the second image data set.

The misaligned image features, that are caused by a misalignment betweenthe registered first and the second image data sets, in the distancedata set are identified. The identification of the misaligned imagefeatures may also include a classification of image features in thedistance data set. For example, artifacts from registering may beidentified as misaligned image features in the distance data set.Identifying the misaligned image features in the distance data set mayinclude a localization and/or marking, for example an annotation, of themisaligned image features in the distance data set. All differencesbetween the registered first image data set and the second image dataset, that are present in the distance data set and are not caused by amisalignment, may be identified as a change at the examination region ofthe examination object, moreover. For this, for example a blood flowsimulation, for example a two-dimensional and/or three-dimensional one,for modeling a contrast medium spread in the examination region of theexamination object may be used. The misaligned image features in thedistance data set may include, for example, geometric image featuresand/or high-contrast objects, for example edges.

Providing the identified misaligned image features may include, forexample, storage on a computer-readable storage medium and/or display ona representation unit and/or transfer to a providing unit. In thisconnection, for example a graphic, for example color-coded, depiction ofthe identified misaligned image features in the distance data set mayfacilitate an evaluation of the result of registering. Intuitivecapturing of the misaligned image features identified in the distancedata set, for example by an operator, may be facilitated by observationof the graphic depiction. The graphic depiction of the misaligned imagefeatures may include a superimposition, for example a weightedsuperimposition, with the registered first image data set and/or thesecond image data set.

According to an embodiment of the computer-implemented method forproviding misaligned image features, at least one parameter ofregistering may be adjusted, for example iteratively, on the basis ofthe misaligned image features identified. A number and/or acharacteristic of misaligned image features in the distance data set maybe reduced by, for example iteratively, repeated execution ofregistering, determining, and identifying. The result of registering inmay be improved hereby.

In an embodiment of the computer-implemented method for providingmisaligned image features, the identification of misaligned imagefeatures may also be based on the first image data set, the second imagedata set and/or the registered first image data set.

For example, the identification of misaligned image features in thedistance data set in may be based on a comparison of the distance dataset with the first image data set, the second image data set and/or theregistered image data set. For example, anatomical and/or geometricimage features, that are present in the first image data set, the secondimage data set and/or the registered image data set, are compared withthe misaligned image features identified in the distance data set. Thismay provide that no anatomical image features and/or changes at theexamination region of the examination object, for example due to aspreading movement of a contrast medium, are identified as misalignedimage features. Artifacts, that may be caused, for example, by are-screening and/or re-scaling of the registered first image data set,may be identified as misaligned image features by consideration of thefirst image data set, moreover.

In an embodiment of the computer-implemented method for providingmisaligned image features, the identification of the misaligned imagefeatures may take place by applying a trained function to input data.The input data may be based on the distance data set. At least oneparameter of the trained function may be based on a comparison oftraining misaligned image features with comparison misaligned imagefeatures.

The trained function may be trained by a method of machine learning. Forexample, the trained function may be a neural network, for example aconvolutional neural network (CNN) or a network including aconvolutional layer.

The trained function maps input data on output data. The output data mayalso depend, for example, on one or more parameter(s) of the trainedfunction. The one or more parameter(s) of the trained function may bedetermined and/or adjusted by training. Determination and/or adjustmentof the one or more parameter(s) of the trained function may be based,for example, on a pair of training input data and associated trainingoutput data, with the trained function for generating training imagingdata being applied to the training input data. For example,determination and/or adjustment may be based on a comparison of thetraining imaging data and the training output data. A trainablefunction, in other words, a function with one or more as yet unadjustedparameter(s), may also be referred to as a trained function.

Other terms for trained function are trained mapping rule, mapping rulewith trained parameters, function with trained parameters, algorithmbased on artificial intelligence, machine learning algorithm. Oneexample of a trained function is an artificial neural network. The edgeweights of the artificial neural network match the parameters of thetrained function. Instead of the term “neural network”, the term“neuronal network” may also be used. For example, a trained function mayalso be a deep neural network (deep artificial neural network). Afurther example of a trained function is a “Support Vector Machine”. Forexample, other machine learning algorithms may also be used as a trainedfunction.

The trained function may be trained, for example, by a back propagation.Training imaging data may be determined by applying the trained functionto training input data. A difference between the training imaging dataand the training output data may be determined by applying an errorfunction to the training imaging data and the training output dataaccordingly. At least one parameter, for example a weighting, of thetrained function, for example of the neural network, may be iterativelyadjusted on the basis of a gradient of the error function in respect ofthe at least one parameter of the trained function. The differencebetween the training mapping data and the training output data duringtraining of the trained function may be minimized hereby.

The trained function, for example the neural network, includes an inputlayer and an output layer. The input layer may be configured to receiveinput data. The output layer may be configured to provide mapping data.The input layer and/or the output layer may each include a plurality ofchannels, for example neurons.

At least one parameter of the trained function may be based on acomparison of training misaligned image features with comparisonmisaligned image features. The training misaligned image features and/orthe comparison misaligned image features may be determined as part of acomputer-implemented method for providing a trained function, that willbe explained in the further course of the description. For example, thetrained function may be provided by an embodiment of thecomputer-implemented method for providing a trained function.

This may provide a robust and reliable identification of the misalignedimage features in the distance data set.

In an embodiment of the computer-implemented method for providingmisaligned image features, the identification of the misaligned imagefeatures may take place by applying a trained function to input data.The input data may also be based on the first image data set, the secondimage data set and/or the registered first image data set.

This may provide that no anatomical image features and/or changes at theexamination region of the examination object, for example due to aspreading movement of a contrast medium, are identified as misalignedimage features. Artifacts, that may be caused, for example, by are-screening and/or re-scaling of the registered first image data set,for example during registering, may be identified as misaligned imagefeatures by consideration of the first image data set.

In an embodiment of the computer-implemented method for providingmisaligned image features, the distance data set may include a vectorfield and/or a tensor field and/or distance information between theregistered first and the second image data set.

The vector field and/or may include a tensor field and/or the distanceinformation may include a mapping rule between individual image points,for example those that correspond with each other, of the registeredfirst and the second image data set. Image points and/or image regionsof the distance data set, to which the vector field and/or the tensorfield and/or the distance information allocates a slight differencebetween the registered first and the registered image data set, may beexcluded in the identification of the misaligned image features. Forexample, a difference between individual image points and/or an imageregion of the registered first image data set and of the second imagedata set may be quantified by the vector field and/or the tensor fieldand/or the distance information. A change at the examination region ofthe examination object, for example a spreading movement of a contrastmedium, may be determined as such by a mapping rule between image pointsof the registered first image data set and image points of the secondimage data set. This may provide that changes at the examination regionof the examination object, that are mapped in the registered first andthe second image data set, are not identified as misaligned imagefeatures in step d). For example, the vector field and/or the tensorfield may quantitatively describe a spatial change in position of ananatomical image feature mapped in at least one image point of theregistered first image data set and at least one further image point ofthe second image data set.

In an embodiment of the computer-implemented method for providingmisaligned image features, the second image data set may be recordedbefore or after the first image data set. The second image data set maymap a change over time at the examination region of the examinationobject compared to the first image data set.

The second image data set may map the at least partially sharedexamination region of the examination object at one instant and/or overa period before or after recording of the first image data set. Forexample, the first and the second image data set may be recordedchronologically. For example, the change at the examination region ofthe examination object may include a movement of a medical instrument,for example a catheter, and/or a spreading movement of a contrastmedium, for example in a vessel system of the examination object.

The first image data set, for example, may map a condition of theexamination region of the examination object before a change at theexamination region. If the first image data set includes a scene, forexample a two-dimensional and/or three-dimensional one, the examinationregion of the examination object may be dynamically mapped over aperiod. The second image data set, if it is recorded after the firstimage data set, may map the change at the examination region. If thesecond image data set includes a scene, for example a two-dimensionaland/or three-dimensional one, the examination region of the examinationobject may be mapped dynamically over a period after recording of thefirst image data set. For example, all dynamically occurring changes andthose that are repeated over time during recording of the first and ofthe second image data sets, for example physiological ones, at theexamination region of the examination object, for example a blood flowand/or a respiratory movement, are considered hereby when determiningthe distance data set. For example, these changes at the examinationregion of the examination object, that are present in the first and thesecond image data sets, may be identified as not being misaligned imagefeatures.

Additional consideration of the first image data set, the second imagedata set and/or the registered first image data set may provide thatchanges at the examination region of the examination object, that arepresent in the first and the second image data sets, are not identifiedas misaligned image features.

In an embodiment of the computer-implemented method for providingmisaligned image features, the first image data set may include firstindividual images of a first phase and the second image data set mayinclude second individual images of a second phase of an image series ofthe examination region of the examination object. The first and thesecond phases may be determined using an acquisition parameter of theimage series and/or a physiological parameter of the examination object.

The first and the second phases of the image series may include, forexample, a temporal and/or physiological phase. The acquisitionparameter of the image series may include, for example, an X-ray doseand/or an exposure time and/or a sensitivity value of an X-ray detectorand/or a dose adjustment of a contrast medium injector and/or a flowrate of a contrast medium injector. The physiological parameter of theexamination object may describe, for example, a heartrate and/or acardiac state and/or a respiratory rate and/or a respiratory state ofthe examination object.

The first and the second phases may be determined using the acquisitionparameter of the image series and/or the physiological parameter of theexamination object in such a way that the individual images of the imageseries are each recorded with substantially the same acquisitionparameters and/or physiological parameters within the first or thesecond phase. For example, all first individual images of the firstphase of the image series may be recorded with substantially the sameacquisition parameters and/or with substantially the same physiologicalparameters of the examination object. Analogously, all second individualimages of the second phase of the image series may be recorded withsubstantially the same acquisition parameters and/or with substantiallythe same physiological parameters of the examination object. In the caseof digital subtraction angiography (DSA), for example the first imagedata set may map a mask phase while the second image data set maps afull phase.

A high consistency between the first individual images of the firstphase and a high consistency between the second individual images of thesecond phase of the image series may be provided. For example, improvedmulti-phase registering between the respective individual images of theimage series is enabled.

In an embodiment of the computer-implemented method for providingmisaligned image features, at least one of the first individual imagesmay be registered with at least one of the second individual images.Registering may also include registering at least some of the firstindividual images with each other and/or registering at least some ofthe second individual images with each other. The distance data set maybe determined on the basis of the registered at least one firstindividual image and the at least one second individual image.Determining may include determining a further distance data set based onthe registered first individual images and/or the registered secondindividual images. In addition, identifying may also include identifyingmisaligned image features in the further distance data set, that arecaused by a misalignment between the registered first individual imagesand/or between the registered second individual images.

For example, movement-induced changes at the examination region of theexamination object within the respective phase of the image series maybe reduced by registering at least some of the first individual imageswith each other and/or at least some of the second individual imageswith each other. The further distance data set may be determined on thebasis of the registered, for example with each other, first individualimages and/or the registered, for example with each other, secondindividual images. For example, the further distance data set mayinclude a first and/or a second distance data set, with the first and/orthe second distance data set each being based on the registered firstand/or the registered second individual images. The first and/or thesecond distance data set may exhibit all differences between theregistered first and/or the registered second individual images. Theregistered first and the registered second individual images within therespective phase of the image series include only slight differences,that may be caused by an acquisition parameter of the image seriesand/or a physiological parameter of the examination object. The furtherdistance data set, for example the first and/or the second distance dataset, may include substantially misaligned image features, that arecaused by a misalignment, for example during registering of therespective individual images in relation to each other.

These misaligned image features may be identified in the furtherdistance data set, for example analogously to the identification of themisaligned image features in the distance data set.

The first individual images registered with each other may also beregistered with at least one of the second individual images, forexample with the second individual images registered with each other,moreover. The distance data set may be determined on the basis of theregistered at least one first individual image and the at least onesecond individual image. Both the misaligned image features in thedistance data set, that are caused by a misalignment between theregistered at least one first individual image and the at least onesecond individual image, as well as the misaligned image features in thefurther distance data set, that are caused by a misalignment between thefirst individual images of the first phase and/or between the secondindividual images of the second phase of the image series, may beidentified hereby.

A graphic, for example color-coded, representation of the furtherdistance data set, for example of the first and/or of the seconddistance data set, and/or the misaligned image features identifiedtherein may be displayed on a representation unit. An evaluation of theresults of registering is provided hereby. Intuitive capturing of themisaligned image features identified in the distance data set and in thefurther distance data set, for example by an operator, may befacilitated by observation of the graphic representation. The graphicrepresentation of the misaligned image features identified in thedistance data set and/or the further distance data set may include asuperimposition, for example a weighted one, with the registered firstimage data set and/or the second image data set. The graphicrepresentation may include a superimposition of the first distance dataset, of the second distance data set and/or of the distance data setwith the misaligned image features identified therein in each case.

A reliable identification of misaligned image features, for example alsowith consideration of acquisition parameters and/or physiologicalchanges at the examination region of the examination object, is providedhereby.

Embodiments include a computer-implemented method for providing atrained function. In a first step, a first and a second training imagedata set are received. The first and the second training image data setsat least partially map a shared examination region of an examinationobject. In a second step, the first training image data set isregistered with the second training image data set. After this atraining distance data set is determined on the basis of the registeredfirst training image data set and the second training image data set. Byway of a comparison of the training distance data set with the firsttraining image data set and/or the second training image data set,comparison misaligned image features in the training distance data setare identified, that are caused by a misalignment between the registeredfirst and the second training image data set. Training misaligned imagefeatures are identified in the training distance data set, that arecaused by a misalignment between the registered first and the secondtraining image data sets, by applying the trained function to inputdata. The input data of the trained function is based on the trainingdistance data set. At least one parameter of the trained function isadjusted on the basis of a comparison of the training misaligned imagefeatures with the comparison misaligned image features. The trainedfunction is provided in accordance with this.

Receiving of the first and of the second training image data sets mayinclude, for example, acquisition and/or reading out from acomputer-readable data memory and/or receiving from a data memory unit,for example a database. The first and the second training image datasets may be provided by a providing unit of a medical imaging device forrecording the first and the second training image data sets. The firstand the second training image data sets map at least partially a sharedexamination region of an examination object.

The first and the second training image data set may include, forexample, all properties of the first and of the second image data sets,that were described in relation to the computer-implemented method forproviding misaligned image features, and vice versa. For example, thefirst training image data set may be a first image data set and thesecond training image data set may be a second image data set sein. Thefirst and the second training image data sets may be simulated.

On registering the first training image data set with the secondtraining image data set the first training image data set may be rigidlyand/or non-rigidly transformed according to a transformation rule. Thefirst training image data set may be registered in relation to thesecond training image data set according to the transformation rule. Thesecond training image data set may be registered in relation to thefirst training image data set according to the transformation rule.Registering may take place in such a way that the section of the atleast partially shared examination region, mapped in the first and inthe second training image data sets, is optimally correlated.Registering the first training image data set with the second trainingimage data set may also take place on the basis of metadata of the firstand/or of the second training image data set.

The training distance data set may be determined on the basis of theregistered first training image data set and the second training imagedata set. The training distance data set may include informationrelating to a difference and/or a distance between at least one firstregion-of-interest (ROI), including a plurality of image points of theregistered first training image data set, and a secondregion-of-interest corresponding therewith, including a plurality ofimage points of the second training image data set. The trainingdistance data set may also include information, for exampletwo-dimensional and/or three-dimensional information, relating to aregistering error between the at least one first region-of-interest andthe second region-of-interest corresponding therewith. Determination ofthe training distance data set may include a difference and/or a scalarproduct and/or a comparison between the registered first training imagedata set and the second training image data set. The training distancedata set may be determined in a standardized manner and/or bestandardized.

The training distance data set may include all differences between theregistered first training image data set and the second training imagedata set.

By way of a comparison of the training distance data set with the firsttraining image data set and/or the second training image data set, thecomparison misaligned image features may be identified in the trainingdistance data set, that are caused by a misalignment between theregistered first and the second training image data sets. Identifyingthe comparison misaligned image features in the training distance dataset may include a localization and/or marking, for example anannotation, of the comparison misaligned image features in the trainingdistance data set. The comparison of the training distance data set withthe first training image data set and/or the second training image dataset may be based, for example, on a comparison of image points and/orimage regions. In addition, the comparison may take place by, forexample manual and/or semi-automatic, annotation of a graphicrepresentation of a, for example weighted, superimposition of thetraining distance data set with the first training image data set and/orthe second training image data set.

For example, all artifacts of registering between the first and thesecond training image data sets may be identified as comparisonmisaligned image features in the training distance data set hereby. Alldifferences between the registered first training image data set and thesecond training image data set, that are present in the trainingdistance data set and are not caused by a misalignment, may beidentified as a change at the examination region of the examinationobject, moreover. For example, a blood flow simulation for modeling acontrast medium spread in the examination region of the examinationobject may be used for this. The comparison misaligned image features inthe training distance data set may include, for example, geometric imagefeatures and/or high-contrast objects, for example edges.

The training misaligned image features in the training distance data setmay be identified by applying the trained function to the input data,that is based on the training distance data set.

At least one parameter of the trained function may be adjusted on thebasis of the comparison between the training misaligned image featuresand the comparison misaligned image features.

The identification of the training misaligned image features by applyingthe trained function to the input data may be improved hereby in such away that changes at the examination region of the examination object,that are not caused by a misalignment between the first training imagedata set and the second training image data set, are not identified astraining misaligned image features.

Providing the trained function may include, for example, storage on acomputer-readable storage medium and/or a transfer to a providing unit.

The method may provide a trained function, that may be used in thecomputer-implemented method for providing misaligned image features.

In an embodiment of the computer-implemented method for providing atrained function, the input data may also be based on the first trainingimage data set, the second training image data set and/or the registeredfirst training image data set.

This may provide that no anatomical and/or geometric image features,that are present in the first, the registered first and/or the secondtraining image data set, are identified as training misaligned imagefeatures.

Embodiments include a providing unit including a computing unit and aninterface. The interface may be configured for receiving a first and asecond image data set. The computing unit may be configured to registerthe first image data set with the second image data set. In addition,the computing unit may be configured for determining a distance data setbased on the registered first image data set and the second image dataset. The computing unit may be configured for identifying the misalignedimage features in the distance data set, with the misaligned imagefeatures being caused by a misalignment between the registered first andthe second image data set. The interface may be configured for providingthe identified misaligned image features.

A providing unit for providing misaligned image features is configuredto carry out the above-described methods for providing misaligned imagefeatures and their aspects. The providing unit is configured to carryout the methods and their aspects in that the interface and thecomputing unit are configured to carry out the corresponding methodsteps.

The advantages of the providing unit for providing misaligned imagefeatures substantially match the advantages of the computer-implementedmethods for providing misaligned image features. Features, advantages,or alternative embodiments mentioned here may likewise be transferred tothe other claimed subject matters, and vice versa.

Embodiments include a medical imaging device including a providing unitfor providing misaligned image features. The medical imaging device, forexample the providing unit, is configured to carry out acomputer-implemented method for providing misaligned image features. Themedical imaging device may be configured, for example, as a medicalX-ray device, for example a C-arm X-ray device, and/or computedtomography system (CT) and/or magnetic resonance system (MRT) and/orsonography system. The medical imaging device may be configured forrecording and/or receiving and/or providing the first and/or the secondimage data set.

The medical imaging device may include, for example, a representationunit, for example a display and/or a monitor, that is configured todisplay information and/or graphic representations of information of themedical imaging device and/or the providing unit and/or furthercomponents. For example, the representation unit may be configured fordisplaying a graphic representation of the first and/or the second imagedata set and/or the misaligned image features.

The advantages of the medical imaging device substantially match theadvantages of the computer-implemented methods for providing misalignedimage features. Features, advantages, or alternative embodimentsmentioned here may likewise be transferred to the other claimed subjectmatters, and vice versa.

Embodiments include a training unit, that is configured to carry out theabove-described computer-implemented methods for providing a trainedfunction and its aspects. The training unit includes a traininginterface and a training computing unit. The training unit is configuredto carry out these methods and their aspects in that the traininginterface and the training computing unit are configured to carry outthe corresponding method steps. For example, the training interface maybe configured for providing the trained function.

Embodiments include a computer program product with a computer program,that may be loaded directly into a memory of a providing unit, includingprogram segments in order to carry out all steps of thecomputer-implemented method for providing misaligned image features whenthe program segments are executed by the providing unit; and/or that maybe loaded directly into a training memory of a training unit, havingprogram segments in order to carry out all steps of the method forproviding a trained function and/or one of its aspects when the programsegments are executed by the training unit.

Embodiments include a computer-readable storage medium on which programsegments, that may be read and executed by a providing unit, are storedin order to carry out all steps of the computer-implemented method forproviding misaligned image features when the program segments areexecuted by the providing unit; and/or on which program segments, thatmay be read and executed by a training unit, are stored in order tocarry out all steps of the method for providing a trained functionand/or one of its aspects when the program segments are executed by thetraining unit.

Embodiments include a computer program or computer-readable storagemedium, including a trained function provided by a computer-implementedmethod or one of its aspects.

An implementation largely in terms of software has the advantage thateven previously used providing units and/or training units may be easilyretrofitted by way of a software update. In addition to the computerprogram, a computer program product may optionally include additionalcomponents, such as, for example, documentation and/or additionalcomponents, as well as hardware components, such as hardware keys(dongles, etc.) in order to use the software.

BRIEF DESCRIPTION OF THE FIGURES

Embodiments are depicted in the drawings and will be described in moredetail below. Identical reference numerals are used in different figuresfor identical features.

FIGS. 1, 2, 3, and 4 depict schematic representations of differentembodiments of a computer-implemented method for providing misalignedimage features.

FIG. 5 depicts a schematic representation of an embodiment of thecomputer-implemented method for providing a trained function.

FIG. 6 depicts a schematic representation of a providing unit accordingto an embodiment.

FIG. 7 depicts a schematic representation of a training unit accordingto an embodiment.

FIG. 8 depicts a schematic representation of a medical C-arm X-raydevice as an example of a medical imaging device according to anembodiment.

DETAILED DESCRIPTION

FIG. 1 schematically depicts an embodiment of the computer-implementedmethod for providing misaligned image features. A first image data setBD1 and a second image data set BD2 may be received REC-BD1-BD2, withthe first BD1 and the second image data set BD2 depicting at leastpartially a shared examination region of an examination object. Thefirst image data set BD1 may be registered REG-BD1-BD2 with the secondimage data set. A registered first image data set REG-BD1 may beprovided in accordance with this. In addition, a distance data set ADmay be determined DET-AD on the basis of the registered first image dataset REG-BD1 and the second image data set BD2. The misaligned imagefeatures FBM, that are caused by a misalignment between the registeredfirst and the second image data sets may be identified ID-FBM in thedistance data set AD. The identified misaligned image features FBM maybe provided PROV-FBM.

The distance data set AD may include a vector field and/or a tensorfield and/or distance information between the registered first REG-BD1and the second image data set BD2.

The second image data set BD2 may be recorded before or after the firstimage data set BD1, for example in the course of digital subtractionangiography (DSA). The second image data set BD2 may map a change overtime at the examination region of the examination object compared to thefirst image data set BD1.

FIG. 2 schematically depicts an embodiment of the computer-implementedmethod for providing misaligned image features. Identification of themisaligned image features ID-FBM may also be based on the first imagedata set BD1, the second image data set BD2 and/or the registered firstimage data set REG-BD1.

According to an embodiment of the computer-implemented method forproviding misaligned image features PROV-FBM, at least one parameter ofregistering REG-BD1-BD2 may be adjusted, for example iteratively, on thebasis of the misaligned image features FBM identified. A number and/or acharacteristic of misaligned image features FBM in the distance data setmay be reduced by, for example iteratively, repeated execution. Theresult of registering may be improved hereby.

In an embodiment of the computer-implemented method for providingmisaligned image features schematically depicted in FIG. 3 , themisaligned image features ID-FBM may be identified by applying a trainedfunction TF-ID-FBM to input data. The input data may be based on thedistance data set AD. At least one parameter of the trained functionTF-ID-FBM may be based on a comparison of training misaligned imagefeatures with comparison misaligned image features.

The input data of the trained function TF-ID-FBM may also be based onthe first image data set BD1, the second image data set BD2 and/or theregistered first image data set REG-BD1.

FIG. 4 schematically depicts an embodiment of the computer-implementedmethod for providing misaligned image features. The first image data setBD1 may include first individual images EB1.1, EB1.2 to EB1.N1 of afirst phase and the second image data set BD2 may include secondindividual images EB2.1, EB2.2 to EB2.N2 of a second phase of an imageseries of the examination region of the examination object. The firstand the second phases may be determined on the basis of an acquisitionparameter of the image series and/or a physiological parameter of theexamination object.

At least one of the first individual images EB1.1 may be registeredREG-EB1.1-EB2.1 in step b) with at least one of the second individualimages EB2.1. A registered first individual image REG12-EB1.1 may beprovided.

Registering may include registering REG1-EB1 at least some of the firstindividual images EB1.1, EB1.2 to EB1.N1 with each other and registeringREG2-EB2 at least some of the second individual images EB2.1, EB2.2 toEB2.N2. For example, all first individual images EB1.1, EB1.2 to EB1.N1may be registered REG1-EB1 with each other. Analogously, all secondindividual images EB2.1, EB2.2 to EB2.N2 may be registered REG2-EB2 witheach other. A further first image data set REG1-BD1 may be provided inaccordance with this, that includes the first individual imagesREG-EB1.1, REG-EB1.2 to REG-EB1.N1 registered with each other. A furthersecond image data set REG2-BD2 may be provided, that includes the secondindividual images REG-EB2.1, REG-EB2.2 to REG-EB2.N2 registered witheach other.

In addition, the distance data set AD may be determined DET-AD on thebasis of the registered at least one first individual image REG12-EB1.1and the at least one second individual image EB2.1. Determining may alsoinclude determining DET-AD1 and DET-AD2 a further distance data set,wherein the further distance data set may include a first distance dataset AD1 and a second distance data set AD2. The further distance dataset, for example the first AD1 and the second distance data set AD2, maybe determined DET-AD1 and DET-AD2 on the basis of the first individualimages REG1-BD1 registered with each other and/or the registered secondindividual images REG2-BD2.

Identifying may include identifying misaligned image features ID-FBM inthe further distance data set, for example the first AD1 and the seconddistance data set AD2, that are caused by a misalignment between thefirst individual images REG1-BD1 registered with each other and/orbetween the second individual images REG2-BD2 registered with eachother.

FIG. 5 schematically depicts an embodiment of the computer-implementedmethod for providing a trained function PROV-TF-ID-FBM. A first TBD1 anda second training image data set TBD2 may be received REC-TBD1-TBD2 in afirst step. The first TBD1 and the second training image data set TBD2may at least partially map a shared examination region of theexamination object. The first training image data set TBD1 may beregistered REG-TBD1-TBD2 with the second training image data set TBD2 ina second step. A registered first training image data set REG-TBD1 maybe provided in accordance with this. A training distance data set TADmay be determined DET-TAD in a third step on the basis of the registeredfirst training image data set TBD1 and the second training image dataset TBD2. Comparison misaligned image features VFBM in the trainingdistance data set TAD, that are caused by a misalignment between theregistered first training image data set REG-TBD1 and the secondtraining image data set TBD2, may be identified ID-VFBM in a fifth step.The comparison misaligned image features ID-VFBM may be identified byway of a comparison of the training distance data set TAD with the firsttraining image data set TBD1 and/or the second training image data setTBD2. In addition, training misaligned image features TFBM in thetraining distance data set TAD, that are caused by a misalignmentbetween the registered first REG-TBD1 and the second training image dataset TBD2, may be identified by applying the trained function TF-ID-FBMto input data. The input data of the trained function TF-ID-FBM may bebased on the training distance data set TAD.

In addition, the input data of the trained function TF-ID-FBM may alsobe based on the first training image data set TBD1, the second trainingimage data set TBD2 and/or the registered first training image data setREG-TBD1.

In a further step, at least one parameter of the trained functionTF-ID-FBM may be adjusted on the basis of a comparison of the trainingmisaligned image features TFBM with the comparison misaligned imagefeatures VFBM ADJ-TF-ID-FBM. The trained function TF-ID-FBM may beprovided PROV-TF-ID-FBM in accordance with this.

FIG. 6 schematically depicts a providing unit PRVS including aninterface IF, a computing unit CU, and a memory unit MU. The providingunit PRVS may be configured to carry out a computer-implemented methodfor providing misaligned image features PROV-FBM and its aspects in thatthe interface IF and the computing unit CU are configured to carry outthe corresponding method steps. The interface IF may be configured, forexample, for receiving the first BD1 and the second image data set BD2.The computing unit CU may be configured for registering REG-BD1-BD2 thefirst image data set BD1 with the second image data set BD2. Inaddition, the computing unit CU may be configured for determining DET-ADa distance data set AD based on the registered first image data setREG-BD1 and the second image data set BD2. The computing unit CU may beconfigured for identifying ID-FBM the misaligned image features FBM inthe distance data set AD, that are caused by a misalignment between theregistered first REG-BD1 and the second image data set BD2. Theinterface IF may be configured for providing PROV-FBM the identifiedmisaligned image features FBM.

FIG. 7 schematically depicts a training unit TRS including a traininginterface TIF, a training computing unit TCU and a training memory unitTMU. The training unit TRS may be configured to carry out acomputer-implemented method for providing a trained function TF-ID-FBMand its aspects in that the training interface TIF and the trainingcomputing unit TCU are configured to carry out the corresponding methodsteps. The training interface TIF may be configured for receiving thefirst TBD1 and the second training image data set TBD2. The trainingcomputing unit TCU may be configured for registering REG-TBD1-TBD2 thefirst training image data set TBD1 with the second training image dataset TBD2. In addition, the training computing unit TCU may be configuredfor determining the training distance data set TAD on the basis of theregistered first training image data set REG-TBD1 and the secondtraining image data set TBD2. The training computing unit TCU may beconfigured for identifying ID-VFBM comparison misaligned image featuresVFBM in the training distance data set TAD, that are caused by amisalignment between the registered first REG-TBD1 and the secondtraining image data set TBD2. The training computing unit TCU may beconfigured for identifying training misaligned image features TFBM inthe training distance data set TAD by applying the trained functionTF-ID-FBM to input data, that is based on the training distance data setTAD. The training computing unit TCU may be configured for adjustingADJ-TF-ID-FBM at least one parameter of the trained function TF-ID-FBMon the basis of a comparison of the training misaligned image featuresTFBM with the comparison misaligned image features VFBM. The traininginterface TIF may be configured for providing PROV-TF-ID-FBM the trainedfunction TF-ID-FBM.

The providing unit PRVS and/or the training unit TRS may be, forexample, a computer, a microcontroller, or an integrated circuit.Alternatively, the providing unit PRVS and/or the training unit TRS maybe a real or virtual network of computers (an English technical term fora real network is “Cluster”, an English technical term for a virtualnetwork is a “Cloud”). The providing unit PRVS and/or the training unitTRS may also be configured as a virtual system, that is implemented on areal computer or a real or virtual network of computers(virtualization).

An interface IF and/or a training interface TIF may be a hardware orsoftware interface (for example PCI bus, USB or Firewire). A computingunit CU and/or a training computing unit TCU may include hardwareelements or software elements, for example a microprocessor or what isknown as an FPGA (acronym for “Field Programmable Gate Array”). A memoryunit MU and/or a training memory unit TMU may be implemented as aRandom-Access Memory (RAM for short) or as a permanent bulk memory (harddisk, USB stick, SD card, Solid State Disk).

The interface IF and/or the training interface TIF may include, forexample, a plurality of sub-interfaces, that execute different steps ofthe respective method. In other words, the interface IF and/or thetraining interface TIF may also be understood as a large number ofinterfaces IF or a large number of training interfaces TIF. Thecomputing unit CU and/or the training computing unit TCU may include,for example, a plurality of sub-computing units, that execute differentsteps of the respective method. In other words, the computing unit CUand/or the training computing unit TCU may also be understood as a largenumber of computing units CU or a large number of training computingunits TCU.

FIG. 8 schematically depicts as an example of a medical imaging device amedical C-arm X-ray device 37. The medical C-arm X-ray device 37 mayinclude a providing unit PRVS for providing misaligned image featuresPROV-FBM. The medical imaging device 37, for example the providing unitPRVS, may be configured for carrying out a computer-implemented methodfor providing misaligned image features PROV-FBM.

The medical C-arm X-ray device 37 includes a detector unit 34 and anX-ray source 33, moreover. For recording the first BD1 and the secondimage data set BD2, for example of at least one projection X-ray imagerespectively, the arm 38 of the C-arm X-ray device 37 may be mounted tomove about one or more axes. The medical C-arm X-ray device 37 mayinclude a movement apparatus 39, that provides a movement of the C-armX-ray device 37 in the space.

For recording the first BD1 and the second image data set BD2 of anexamination region to be mapped of an examination object 31 arranged ona patient supporting facility 32, the providing unit PRVS may send asignal 24 to the X-ray source 33. The X-ray source 33 may then emit anX-ray beam bundle, for example a cone beam and/or fan beam and/orparallel beam. When the X-ray beam bundle, following an interaction withthe region of the of the examination object 31 to be mapped, strikes asurface of the detector unit 34, the detector unit 34 may send a signal21 to the providing unit PRVS. The providing unit PRVS may receive thefirst BD1 and the second image data set BD2, for example with the aid ofthe signal 21.

The medical C-arm X-ray device 37 may include an input unit 42, forexample a keyboard, and/or a representation unit 41, for example amonitor and/or display. The input unit 42 may be integrated in therepresentation unit 41, for example in the case of a capacitive inputdisplay. Control of the medical C-arm X-ray device 37, for example ofthe computer-implemented method for providing misaligned image featuresPROV-FBM, may be provided by an input by an operator at the input unit42. For this, the input unit 42 may send, for example, a signal 26 tothe providing unit PRVS.

The representation unit 41 may be configured to display informationand/or graphic representations of information of the medical imagingdevice 37 and/or the providing unit PRVS and/or further components. Forthis, the providing unit may, for example, send a signal 25 to therepresentation unit 41. For example, the representation unit 41 may beconfigured for displaying a graphic representation of the first BD1and/or the second image data set BD2 and/or the registered first imagedata set REG-BD1 and/or the misaligned image features FBM. A graphic,for example color-coded, representation of the distance data set ADand/or of the misaligned image features FBM identified therein may bedisplayed on the representation unit 41. The graphic representation ofthe misaligned image features FBM may include a superimposition, forexample a weighted one, with the registered first image data set REG-BD1and/or the second image data set BD2.

The schematic representations contained in the described figures do notindicate any scale or size ratio.

Reference is made once again to the fact that the methods described indetail above and the depicted apparatuses are merely preferred exemplaryembodiments that may be modified in a wide variety of ways by a personskilled in the art without departing from the scope of the invention.Furthermore, use of the indefinite article “a” or “an” does not precludethe relevant features from also being present several times. Similarly,the terms “unit” and “element” do not preclude the relevant componentsfrom including a plurality of cooperating sub-components, that,optionally, may also be spatially distributed.

It is to be understood that the elements and features recited in theappended claims may be combined in different ways to produce new claimsthat likewise fall within the scope of the present invention. Thus,whereas the dependent claims appended below depend from only a singleindependent or dependent claim, it is to be understood that thesedependent claims may, alternatively, be made to depend in thealternative from any preceding or following claim, whether independentor dependent, and that such new combinations are to be understood asforming a part of the present specification.

While the present invention has been described above by reference tovarious embodiments, it may be understood that many changes andmodifications may be made to the described embodiments. It is thereforeintended that the foregoing description be regarded as illustrativerather than limiting, and that it be understood that all equivalentsand/or combinations of embodiments are intended to be included in thisdescription.

The invention claimed is:
 1. A computer-implemented method for providing misaligned image features, the method comprising: receiving a first image data set and a second image data set, wherein the first image data set and the second image data set map at least partially a shared examination region of an examination object; registering the first image data set with the second image data set; determining a distance data set based on the registered first image data set and the second image data set; identifying the misaligned image features in the distance data set that are caused by a misalignment between the registered first image data set and the second image data set by applying a trained function to input data that is based on the distance data set, wherein at least one parameter of the trained function is based on a comparison of training misaligned image features with comparison misaligned image features; and providing the identified misaligned image features.
 2. The computer-implemented method of claim 1, wherein the identification of the misaligned image features is also based on at least one of the first image data set, the second image data set, or the registered first image data set.
 3. The computer-implemented method of claim 1, wherein the input data is also based on at least one of the first image data set, the second image data set, or the registered first image data set.
 4. The computer-implemented method of claim 1, wherein the distance data set comprises at least one of a vector field, a tensor field, or distance information between the registered first image data set and the second image data set.
 5. The computer-implemented method of claim 1, wherein the second image data set is recorded after the first image data set, wherein the second image data set maps a change over time at the examination region of the examination object compared to the first image data set.
 6. The computer-implemented method of claim 1, wherein the first image data set comprises first individual images of a first phase and the second image data set comprises second individual images of a second phase of an image series of the examination region of the examination object; wherein the first and the second phases are determined using an acquisition parameter of the image series or a physiological parameter of the examination object.
 7. The computer-implemented method of claim 6, wherein for registering at least one of the first individual images is registered with at least one of the second individual images, wherein registering further comprises registering at least some of the first individual images with each other or registering at least some of the second individual images with each other; wherein the distance data set is determined on a basis of the registered at least one first individual image and the at least one second individual image; wherein determining further comprises determining a further distance data set based on the registered at least one first individual image, the registered at least one second individual image, or the registered at least one first individual image and the registered at least one second individual image; wherein identifying further comprises identifying misaligned image features in the further distance data set that are caused by a misalignment between the registered first individual images, between the registered second individual images, or between the registered first individual images and between the registered second individual images.
 8. A computer-implemented method for providing a trained function, the method comprising: receiving a first training image data set and a second training image data set, wherein the first training image data set and the second training image data set map include at least partially a shared examination region of an examination object; registering the first training image data set with the second training image data set; determining a training distance data set based on the registered first training image data set and the second training image data set; identifying comparison misaligned image features in the training distance data set that are caused by a misalignment between the registered first training image data set and the second training image data set by a comparison of the training distance data set with the first training image data set, the second training image data set, or the first training image data set and the second training image data set; identifying training misaligned image features in the training distance data set that are caused by a misalignment between the registered first training image data set and the second training image data set by applying the trained function to input data, wherein the input data is based on the training distance data set; adjusting at least one parameter of the trained function based on a comparison of the training misaligned image features with the comparison misaligned image features; and providing the trained function.
 9. The computer-implemented method of claim 8, wherein the input data is also based on at least one of the first training image data set, the second training image data set, or the registered first training image data set.
 10. A medical imaging system comprising: a medical imaging device configured to record and provide a first image data set and a second image data set to a providing unit; and the providing unit configured to: register the first image data set with the second image data set; determine a distance data set based on the registered first image data set and the second image data set; apply a trained function to input data that is based on the distance data set, wherein the trained function is configured to identify the misaligned image features that are caused by a misalignment between the registered first training image data set and the second training image data set, wherein at least one parameter of the trained function is based on a comparison of training misaligned image features with comparison misaligned image features; and provide the identified misaligned image features.
 11. The medical imaging system of claim 10, wherein the identification of the misaligned image features is also based on at least one of the first image data set, the second image data set, or the registered first image data set.
 12. The medical imaging system of claim 10, wherein the input data is also based on at least one of the first image data set, the second image data set, or the registered first image data set.
 13. The medical imaging system of claim 10, wherein the distance data set comprises at least one of a vector field, a tensor field, or distance information between the registered first and the second image data set.
 14. The medical imaging system of claim 10, wherein the second image data set is recorded after the first image data set, wherein the second image data set maps a change over time at an examination region of an examination object compared to the first image data set.
 15. The medical imaging system of claim 10, wherein the first image data set comprises first individual images of a first phase and the second image data set comprises second individual images of a second phase of an image series of the examination region of the examination object; wherein the first and the second phases are determined using an acquisition parameter of the image series or a physiological parameter of the examination object.
 16. The medical imaging system of claim 15, wherein the providing unit is configured to register at least one of the first individual images with at least one of the second individual images, wherein registering further comprises registering at least some of the first individual images with each other or registering at least some of the second individual images with each other; wherein the distance data set is determined on a basis of the registered images; wherein determining further comprises determining a further distance data set based on the registered images; wherein identifying further comprises identifying misaligned image features in the further distance data set that are caused by a misalignment between the registered images. 